Hotspots_report
Paulo Manrique, Ashley Farley, Brooke Whittemore, Lindsey Wu, Lisa Prach, Henry Ntuku, Jenny Smith, Adam Bennett, Munyaradzi Tambo, Allison Golden, Inh Kyung Jang, Gonzalo Domingo, Bryan Greenhouse, Petrina Uusiku, Chris Drakeley, Immo Kleinschmidt, Davis Mumbengegwi, Roly Gosling, Michelle S. Hsiang.
September 21th, 2020
Performance evaluation of standard and ultra-sensitive malaria rapid diagnostic tests to detect hotspots of Plasmodium falciparum transmission in the low endemic setting of Zambezi Region, Namibia.
Background: (TBD)
Objective:
To compare the performance of Ultra sensitive rapid diagnostic test (uRDT) and rapid diagnostic test (RDT) in detecting Plasmodium falciparum transmission hotspots at enumeration area level (or eaid) and household (hhid) and household (hhid).
Methods:
The study population included individuals residing within the catchment areas of eleven health facilities in western Zambezi Region (Fig. 2). Specifically, 56 of 102 census enumeration areas (eaids) in the region were initially included based on reliability of incidence data from 2012 to 2014 and presence of incident cases during that time. eaids with large geographical size and having more than 2 borders with other contiguous eaids were dropped (Fig. 1 and Fig. 2). For the household level data (hhid), inclusion criteria included residence in randomly selected households or boarding schools within one of the eaids with around 25 households selected per eaid. For individuals residing within the household (iid), inclusion criteria included head of household consent, and age greater than six months. Exclusion criteria included not having slept in the household or boarding school at least three nights per week in the previous four weeks and refusal to participate.
Figure 1: Study design flowchart and criteria for hotspot definition.
Figure 2: Study site map.
Field procedures:
Questionnaires were administered to consenting subjects from selected households and schools using tablet computers and blood testing and collection was conducted. A household level questionnaire captured geolocation, household population, insecticide-treated bed net ownership, and indoor residual spray coverage. The individual level questionnaire captured demographic data. In addition, axillary temperature of each participant was measured and recorded. Written individual informed consent was obtained from adults, parental consent was obtained for children under 18 years old, and adolescent assent was obtained from minors 12-17 years old. To enroll as many individuals as possible, all households and schools were visited twice. Surveys were conducted in the local language of siLozi. A trained study nurse collected whole blood for malaria testing using a finger prick for each individual in the cross-sectional analysis.
Laboratory or testing methods:
The collected blood was used to assess the presence of current or recent infection of parasites causing Malaria using 1) the rapid diagnostic test (RDT) CareStartTM Malaria HRP2/pLDH (Pf/PAN) Combo test (AccessBio, Somerset, NJ, USA); 2) the Alere Malaria Ag P.f RDT Ultra Sensitive (SD/Alere, Yongin-si, Republic of Korea) (uRDT); 3) a quantitative real-time polymerase chain reaction (qPCR); 4) and the Q-plex ELISA method for HRP2 quantification (HRP2). The realization of RDT and uRDT was done following the manufacturers instructions. Whole blood collected in the cross-sectional survey was centrifuged, and packed red blood cells were stored at -20°C before undergoing DNA extraction using the Quick-DNATMminiprep kit (Zymo Research Corp, Irvine, CA, USA) and a qPCR targeting the varATS region28using template DNA corresponding to 10 μL of whole blood. Samples were considered qPCR positive if parasite density was greater than 0.1 p/μL. HRP2 concentration was determined using the Q-plex ELISA. All parasite density estimates for this study were determined by averaging duplicate runs by qPCR and described in p/μL. Samples were considered Q-plex HRP2 positive if HRP2 concentration was greater than the Q-plex assay LOD of 2.3 pg/mL.
Statistical analysis:
Individual demographic information (iid, hhid, eaid and others), and the laboratory results of the performed diagnostic tests (positive and negative for the four test performed) were entered and stored in .dta format in STATA (version 13; STATA Corp., College Station, TX, USA) and then analyzed in R version 4.0.1 (R Foundation for Statistical Computing., Vienna, Austria). To evaluate the performance of RDTs and uRDTs in detecting hotspots, data was collapsed at enumeration area (eaid) and household (hhid) level.
At eaid level incidence data (cases/1000 person years) was obtained by passive case detection through RDT and microscopy for patients who presented at health facilities during the broader trial study period. Each positive case was documented along with the individual’s eaid. Prevalence by HRP2 and qPCR (reference tests), and by RDTs and uRDTs in each eaid was also measured for all participants. eaids were classified as hotspots based on 6 Criteria: 1) being in the top quartile of HRP2 prevalence, or having a qPCR prevalence above 10%, or an incidence above 50 cases/1000 persons years; 2) being in the top quartile for any of the three metrics mentioned previously; 3) using a hierarchical clustering method to classify the eaids based on their incidence and their prevalence by HRP2 and qPCR tests; 4) being in the top quartile of the incidence; 5) being in the top quartile of the HRP prevalence; and 6) being in the top quartile of the qPCR prevalence (Fig. 1).
For the hierarchical clustering analysis, the best distance (euclidean, manhattan, canberra, binary, and minkowski) and the best agglomerative (ward.D, ward.D2, single, complete, average, and mcquitty) methods were selected based on the maximization of the agglomerative coefficient. The optimum number of clusters was evaluated from k = 2 to 15, and the best value of k was selected based on the minimization of the SD index (sum of the scattering within clusters and distance between clusters) proposed by Halkidi et al. 2000. Results were shown ussing a clustered heatmap, and the eaids belonging to the clusters with the highest incidence and prevalence that comprise the 75th quartile of the total eaids were defined as hotspots.
At household level the number of screened iids and infected iids per house was counted by each test (HRP2, qPCR, RDTs and uRDTs), and hotspots households were defined based on three criteria: 1) the number of individuals detected infected by HRP2 (grater than 2 infected iids in the household) or by qPCR (grater than 1 infected iids in the household) tests; 2) the number of individuals detected infected by HRP2 (grater than 2 infected iids in the household) test; and 3) the number of individuals detected infected by qPCR (grater than 1 infected iids in the household) test (Fig. 1).
Performance of RDTs and uRDTs as predictors for hotspots was assessed by an empirical receiver operating characteristic (ROC) curve analysis implemented in the R package pROC. Two-sided 95% confidence intervals for the area under the curve (AUC) were measured base on DeLong method and the selection of the best threshold for predicted variables (prevalence by RDTs and uRDTs) was defined by the maximization of Youden index and its two-sided 95% confidence intervals were determined using an stratified bootstrap with 10,000 samples. Sensitivity, Positive and Negative predicted values and accuracy were also measured for a fixed specificity at 85%, and their two-sided 95% confidence intervals were estimated using the Wilson score method implemented in the Hmisc R package. Statistical differences in the areas under the curves between the different tests (RDTs vs uRDTs) were assessed by a DeLong test and their corresponding statistical power was calculated.
Results:
Study sample and tests results
Fifty-two out of the 56 eaids meet the inclusion criteria, and along these eaids 3699 individuals or iids (in 1011 households or hhids) where screened by RDT, uRDT, qPCR and HRP2 tests (Sup. Table 1). A median of 63.5 iids (IQR = 39.2, range from 25 to 150) and 18 hhids (IQR = 7.25, range from 7 to 36) were screened in the 52 eaids, and a median of 3 iids (IQR = 3, range from 1 to 73) were screened per hhid (Fig. 3 and ST2). Three hundred seventy five individuals were positive for any of the 4 test performed (278 for HRP2, 134 for qPCR, 89 for RDTs and 111 for uRDTs) (ST3), and 246 households had positive individuals for any of these tests (204 by the HRP2, 87 by qPCR, 76 by RDTs, and 92 households by uRDTs) (ST3).
Figure 3: Distribution of the A) number screened individuals (iid) and B) the number of screened household (hhid) per enumeration area (eaid); C) distribution of the number of screened individuals per household; and D) number of positive iids by test.
Performance of RDTs and uRDTs at eaid level
In the tests used as reference to define hotspots (HRP2 and qPCR), the median prevalence by HRP2 and qPCR were 6.35 (IQR = \(\pm\) 6.09, range from 0.9 to 30.6) and 1.8 (IQR = \(\pm\) 4.08, and range from 0 to 44.7), respectively (Fig. 4A and ST4). Moreover, the median incidence in the eaids was 20.52 cases/1000 persons years (\(\pm\) 36.19 IQR, range from 0 to 199.95). Regarding RDTs and uRDTs, the median prevalence were 1.98 (\(\pm\) 3.18 IQR), and 2.35 (\(\pm\) 4.29 IQR), respectively (Figure 4B, ST4).
Figure 4: A) Jitter dot plot representing the distribution of the incidence, the prevalence by HRP2 and qPCR between hotspots (dark red dots) and not hotspots (dark blue dots) eaids. The red horizontal line represents the cutoff used to define hotspots (eaids with Incidence > 50/1000py, HRP2 prevalence > 75% quantile, or qPCR prevalence > 10%). B) Jitter dot plot representing the distribution of the prevalence in each eaid by RDTs and by uRDTs. Each dot represents an eaid, and gray dots represents the total population screened.
Regardless of the criteria used to select hotspots, uRDTs had greater areas under the curve (AUCs) than RDTs at eaid level. Using the first criterion (Incidence > 50 py, prevalence by HRP2 > 75% quartile, or prevalence by qPCR > 10%), 19 eaids were defined as hotspots. The AUC of the prevalence by uRDTs (AUC = 0.844, 95%CI 0.724 - 0.964) was statistically greater than its counterpart using RDTs (AUC = 0.758, 95%CI 0.612 - 0.903) (p-value < 0.05, Figure 5, ST5). Its sensitivity, accuracy, ppv, and npv at the fixed specificity of 85% were 68.4%, 78.9%, 72.4% and 82.4%, respectively (Figure 5B.1). In the case of the prevalence by RDTs, the sensitivity, accuracy, ppv, and npv were 52.6%, 73.2%, 66.9% and 75.7%, respectively.
Similar results were observed when using the hierarchical clustering method to define hotspots. A dendrogram based on Manhattann distance and the agglomerative Ward.D method was build, and 6 clusters (SD index for K6 = 2.65) were found (SF1 and SF2). Clusters 1 - 4 comprised the 75th quartile of the total eaids, thus 15 eaids were defined as hotspots. The AUC for uRDTs was 0.86 (95%CI 0.75 - 0.97) and it was statistically greater than RDTs (0.74). For uRDTs, the sensitivity, accuracy, ppv, and npv were 73.3%, 81.6%, 66.5%, and 88.7%, respectively. Regarding RDTs, the sensitivity, accuracy, ppv, and npv were 53.3%, 75.9%, 59%, and 81.8%, respectively. Results for the others criteria are found in SF3 - SF4 and ST5 - ST6.
Figure 5: Performance of RDTs and uRDTs in detecting eaid hotspots using Criterion 1. A) ROC curve, solid black points represent the Sensitivity at a fixed specificity of 85%. B) Forest plot comparing the performance of RDTs and uRDTs in terms of Accuracy, Area under the curve, Negative and Positive predicted values, Sensitivity, and Specificity.
Performance of RDTs and uRDTs at hhid level
Because of the presence of outliers in the number of iids pr hhid (more than 60 iids pr hhid) and the large number of households with just one individual screened, only households with more than 2 and less than 20 screened individuals were included to analyze the performance of RDTs and uRDTs at household level. For all tests, a median of one infected individual per household was found, and a total of 227/815 households had at least one individual detected infected for any of the four tests (191 by HRP2, 81 by qPCR, 71 by RDTs, and 87 by uRDTs).
Figure 6: Jitter dot plot representing the distribution of the number of infected individuals per houshold by each test and between hotspots (dark red dots) and not hotspots (dark blue dots) hhids. Each dot represents a hhid, and gray dots represents the total households screened..
One hundred twelve hhids were classified as hotspots and the AUC for uRDTs was 0.705, while the AUC for RDTs was 0.704 (p-value = 0.5, power = 0.07) (ST5). Additionally, none of the tests had a good performance in terms of sensitivity, 45.5% (CI95% 36.6 - 54.8) for uRDTs and 43.8% (CI95% 34.9 - 53) for RDTs (Fig. 7B, ST6). Performance using thee other two criteria are shown in SF5.
Figure 7: Performance of RDTs and uRDTs in detecting hhid hotspots using households with individuals positives to HRP2 as reference. A) ROC curve, solid black points represent the Sensitivity and Specificity that maximize the Youden metric. B) Forest plots comparing the performance of RDTs and uRDTs in terms of Accuracy, Area under the curve, Negative and Positive predicted values, Sensitivity, and Specificity.
Acknowledgements
We thank study participants, and support from field staff, Ministry of Health and Social Services, and others on the study team (Mi-suk Kang Dufour, Leah Schrubbe, Joy Yala, Sofonias Tessema).
Supplementary information
###Supplementary Figure 1
Supplementary Figure 1: Majority rule to define optimal number of clusters. x-axis represents the number of clusters present in the data set, and the y-axis the number of criteria proposing an specific number of clusters.
###Supplementary Figure 2
Supplementary Figure 2: Clustered Heatmap representing the similarities in the Incidence, the HRP2 prevalence and the qPCR prevalence among communities. Bottom clusters in the y dendrogram were used to define eaids that are hotspots.
###Supplementary Figure 3
Supplementary Figure 3: Performance of RDTs and uRDTs in detecting eaid hotspots using Criterion 2 (A and B) and Criterion 3 (Hierarchical clustering or HC, C and D). A - C) ROC curve, solid black points represent the Sensitivity at the fixed specificity of 85%. B - D) Forest plot comparing the performance of RDTs and uRDTs in terms of Accuracy, Area under the curve, Negative and Positive predicted values, Sensitivity, and Specificity.
###Supplementary Figure 4
Supplementary Figure 4: Performance of RDTs and uRDTs in detecting eaid hotspots using Criterion 4 (Incidence, A and B), 5 (HRP2, C and D) and 6 (qPCR, E and F). A, C and E) ROC curve, solid black points represent the Sensitivity at a fixed specificity of 85%. B, D and F) Forest plot comparing the performance of RDTs and uRDTs in terms of Accuracy, Area under the curve, Negative and Positive predicted values, Sensitivity, and Specificity.
###Supplementary Figure 5
Supplementary Figure 5: Performance of RDTs and uRDTs in detecting hhid hotspots using Criterion 3 (HRP2, A and B), and 4 (qPCR, C and D). A, and C) ROC curve, solid black points represent the Sensitivity and Specificity that maximize the Youden metric. B, and D) Forest plot comparing the performance of RDTs and uRDTs in terms of Accuracy, Area under the curve, Negative and Positive predicted values, Sensitivity, and Specificity.
###Supplementary Table 1| Individuals | Households | Enumeration areas |
|---|---|---|
| 3699 | 1011 | 52 |
| test | median | IQR | min | max |
|---|---|---|---|---|
| iid/eaid | 63.5 | 39.25 | 25 | 150 |
| hhid/eaid | 18.0 | 7.25 | 7 | 36 |
| iid/hhid | 3.0 | 3.00 | 1 | 73 |
| test | result | n.iid | n.hhid |
|---|---|---|---|
| Total | Positive | 375 | 246 |
| HRP2 | Positive | 278 | 204 |
| qPCR | Positive | 134 | 87 |
| RDT | Positive | 89 | 76 |
| uRDT | Positive | 111 | 92 |
| metrics | mean | ci95 | median | IQR | min | max | Group |
|---|---|---|---|---|---|---|---|
| Incidence | 33.849 | 11.720 | 20.515 | 36.188 | 0.000 | 199.954 | Reference |
| prev.HRP2 | 7.685 | 1.505 | 6.349 | 6.088 | 0.901 | 30.612 | Reference |
| prev.qPCR | 3.650 | 1.808 | 1.802 | 4.082 | 0.000 | 44.681 | Reference |
| prev.RDT | 2.488 | 0.906 | 1.985 | 3.175 | 0.000 | 20.408 | Test |
| prev.uRDT | 3.016 | 0.884 | 2.353 | 4.288 | 0.000 | 16.327 | Test |
| prop.RDT.1 | 0.072 | 0.019 | 0.059 | 0.108 | 0.000 | 0.294 | Test |
| prop.uRDT.1 | 0.087 | 0.021 | 0.076 | 0.136 | 0.000 | 0.308 | Test |
| Level | response | comparison | auc1 | auc2 | difference | p.value | power |
|---|---|---|---|---|---|---|---|
| eaid | Criterion1 | prev.uRDT-prev.RDT | 0.8437002 | 0.7575758 | 0.0861244 | 0.0344611 | NaN |
| eaid | Criterion1.2 | prev.uRDT-prev.RDT | 0.7874074 | 0.7325926 | 0.0548148 | 0.1372161 | 0.5935168 |
| eaid | Criterion1.HRP2 | prev.uRDT-prev.RDT | 0.8284024 | 0.7218935 | 0.1065089 | 0.0264135 | NaN |
| eaid | Criterion1.Incidence | prev.uRDT-prev.RDT | 0.7504931 | 0.7919132 | -0.0414201 | 0.2894520 | 0.0692939 |
| eaid | Criterion1.qPCR | prev.uRDT-prev.RDT | 0.6676529 | 0.6775148 | -0.0098619 | 0.8225670 | 0.0110549 |
| eaid | Criterion2 | prev.uRDT-prev.RDT | 0.8603604 | 0.7450450 | 0.1153153 | 0.0104775 | NaN |
| hhid | hh_hotspots1 | uRDT-RDT | 0.7050460 | 0.7043601 | 0.0006858 | 0.9519697 | 0.0213430 |
| hhid | hh_hotspots2 | uRDT-RDT | 0.7705540 | 0.7459295 | 0.0246245 | 0.2622250 | 0.2043600 |
| hhid | hh_hotspots3 | uRDT-RDT | 0.7102886 | 0.7195563 | -0.0092677 | 0.4756426 | 0.0903953 |
###Supplementary Table 6
###Supplementary Table 7
###Supplementary Table 8| test | median | q25 | q75 | IQR | min | max | pos.hhid |
|---|---|---|---|---|---|---|---|
| scr.iid | 4 | 3 | 5 | 2 | 2 | 17 | 227 |
| HRP2 | 1 | 1 | 2 | 1 | 1 | 5 | 191 |
| qPCR | 1 | 1 | 2 | 1 | 1 | 8 | 81 |
| RDT | 1 | 1 | 1 | 0 | 1 | 4 | 71 |
| uRDT | 1 | 1 | 1 | 0 | 1 | 4 | 87 |